Target follow-up device and camera comprising the same

ABSTRACT

The target follow-up device of the present invention is adapted quickly and reliably to follow up the position of a target which is moving at high speed. It incorporates an imaging sensor having a number of imaging elements, which performs photometry over the photographic field and which outputs input image data. Based upon this imaging sensor output, input image data for the target to be followed up are stored as reference image data by a memory device. Based upon minimum residue calculation between the input image data and the reference image data, the target follow-up device of the present invention determines the position of the target. In this determination operation, the target follow-up device selects at least one color from a plurality of colors contained in common by the input image data and the reference image data, and calculates the minimum residue for the selected color component.

This is a Division of application Ser. No. 08/114,148 filed Sep. 1,1993, now U.S. Pat. No. 5,392,088.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a target follow-up device which followsup a predetermined target in the photographic field of an imaging devicesuch as a camera or the like, and also relates to a camera equipped withsuch a target follow-up device.

2. Description of the Related Art

Such a type of target follow-up device has for example been described inJapanese Patent Publications 60-33350 and 61-19076. In these prior artdevices, the correlation coefficient between a previously storedreference image and an input image obtained by A/D conversion of apicture signal supplied by an imaging device is calculated, and thepredetermined target is followed up by determining which coordinatesyield the maximum value of this correlation coefficient. In this case,with the intention of being able to perform follow up action in realtime, a region for calculation of the correlation coefficient isestablished in the photographic field, and the calculation of thecorrelation coefficient with the reference image is performed over thisregion.

Further, there is a per se known type of device (for example, refer toU.S. Pat. No. 5,031,049) which follows up a target (the object to bephotographed) within the photographic field by detecting a colordifference signal over the photographic field, recording this colordifference signal over a designated area (the area which thephotographer designates as containing the principal target), comparingthis recorded color difference signal with the color difference signalin the periphery of the designated area, and thereby determining to whatposition within the photographic field the principal target has moved.

Nevertheless, with these prior art type target follow-up devices, sincethe region for correlation calculation either was set by taking as itscenter the position of the target as determined in the previousiteration of the target follow-up process, or simply was a fixed areawithin the photographic field of view, therefore there has been a riskthat, when photographing a sports scene or the like containing a targetwhich is moving very quickly, the target may move out of the region forcorrelation calculation, which causes problems with regard to followingup the position of the target. Further, when the shape of the targetchanges quickly, as again can easily occur when photographing a sportsscene or the like (for example, when the target rotates quickly from afull frontal aspect to a profile aspect), or when the brightness of thetarget changes quickly (for example when it moves from a sunlit area toa shaded area), then in some cases it becomes impossible to performaccurate target follow-up by calculation of the correlation coefficientwith reference image data recorded in advance.

Further, up to the present no camera has existed in which the operationof the above type of target follow-up device has been organicallycorrelated with photographic modes, focus detection modes, or auto zoommodes of the camera.

Now, the expression "photographic mode" is used in this specification inthe following manner, for example: the most suitable mode of exposurecontrol for performing sports photography is termed the "sportsphotography mode", the most suitable mode of exposure control forperforming portrait photography is termed the "portrait photographymode", etc. Further, the expression "focus detection mode" is used inthis specification in the following manner, for example: a mode in whichfocusing adjustment movement of the photographic lens is repeatedlyperformed based upon the output of a focus detection device is termedthe "continuous mode"; a mode in which, when once based upon the outputof a focus detection device the condition in which of the photographiclens is on focus is detected, afterwards the driving of the photographiclens is forcibly stopped is termed the "one shot mode", a mode in whichthe focus can be independently detected for a plurality of regions inthe photographic field of view based upon the output of a focusdetection device is termed the "multi mode", etc. Further, with anautomatic zoom lens, there are so called "auto zoom" modes like one inwhich the zoom lens is automatically driven so as always to maintain aphotographic magnification determined in advance, even in the event ofchange in the photographic distance or the like, etc.

SUMMARY OF THE INVENTION

An object of the present invention is to provide a target follow-updevice which can quickly, accurately, and reliably follow up theposition of a target which is moving at high speed.

Another object of the present invention is to provide a camera equippedwith a target follow-up device, in which the operability of the camerais enhanced by the provision of organic liaison between the operation ofthe target follow-up device and existing camera modes such as thephotographic mode, the focus detection mode, the auto zoom mode, etc.

In order to attain these objects there is proposed, according to thepresent invention as expressed in one manner, a target follow-up device,having: an imaging means which outputs input image data having aplurality of color components; a memory means which, based upon theoutput from said imaging means, records image data of a target to befollowed up as reference data; a calculation means which calculates theamount of correlation such as a minimum residue between said input imagedata and said reference image data; and a position detection meanswhich, based upon said minimum residue, detects the position of saidtarget; comprising a color selection means which selects at least onecolor from a plurality of colors contained in common by said input imagedata and said reference image data; wherein said calculation meanscalculates said minimum residue for the color component selected by saidcolor selection means.

Further, according to the present invention as expressed in anothermanner, there is proposed a target follow-up device, comprising: animaging means which outputs input image data; a memory means whichsamples a portion of said input image data which is to be used as imagedata of a target to be followed up, and records it as reference data; acalculation means which repeatedly calculates the minimum residuebetween said reference image data and said input image data in acalculation region wider than the region which includes said referenceimage data set within said photographic field; and a position detectionmeans which, based upon said minimum residue, detects the position ofsaid target; and a region setting means which sets said calculationregion to be used for the next iteration, based upon said minimumresidue which was calculated by said calculation means.

Further, according to the present invention as expressed in anothermanner, there is proposed a target follow-up device, having: an imagingmeans which outputs input image data; a memory means which, based uponthe output from said imaging means, records as reference data image dataof a target to be followed up; a calculation means which calculates theminimum residue between said reference image data and said input imagedata; and a position detection means which, based upon said minimumresidue, detects the position of said target; comprising a refreshingmeans which, each time the detection of the position of said target isperformed by said position detection means, samples at least a portionof said input image data according to the position of said target, andrefreshes said reference image data for the next iteration using saidinput image data thus calculated; and wherein said calculation meanscalculates said minimum residue using the reference image data refreshedby said refreshing means.

Further, according to the present invention as expressed in anothermanner, there is proposed a target follow-up device, having: an imagingmeans which outputs input image data; a memory means which, based uponthe output from said imaging means, records as reference data image dataof a target to be followed up; a calculation means which calculates theminimum residue between said reference image data and said input imagedata; and a position detection means which, based upon said minimumresidue, detects the position of said target; comprising a movementdetection means which detects the movement of said target based upon atleast two of the positions of said target detected successively by saidposition detection means, and a position prediction calculation meanswhich calculates a predicted position of said target from the movementof said target detected by said movement detection means.

Further, according to the present invention as expressed in anothermanner, there is proposed a camera having a target follow-up devicewhich follows up the movement of an object to be photographed,comprising: a photographic mode setting means which possesses aplurality of photographic modes for performing suitable exposure controlaccording to photographic circumstances, and which includes among saidplurality of photographic modes a moving target photographic mode whichis suitable for when said object to be photographed is moving; and acontrol means which starts said target follow-up device when said movingbody photographic mode is set by said photographic mode setting means.

Further, according to the present invention as expressed in anothermanner, there is proposed a camera having a target follow-up devicewhich follows up the movement of an object to be photographed,comprising: a follow-up starting means which starts said targetfollow-up device; a photographic lens; a drive means which drives saidphotographic lens; a focusing detection means which detects the focusadjustment condition of said object to be photographed; a focusdetection mode changeover means which is operated to change over betweena continuous mode in which focus adjustment movement of saidphotographic lens is repeatedly performed by said drive means based uponthe output of said focusing detection means, and a one shot mode inwhich, when once the focus adjustment condition of said photographiclens has been detected, based upon the output of said focusing detectionmeans, subsequently the movement of said photographic lens by said drivemeans is compulsorily stopped; and a control means which changes overfrom said one shot mode to said continuous mode when said targetfollow-up device is started by said follow-up starting means, even ifsaid one shot mode is selected by said focus detection mode changeovermeans.

Further, according to the present invention as expressed in anothermanner, there is proposed a camera having a target follow-up devicewhich follows up the movement of an object to be photographed,comprising: a follow-up starting means which starts said targetfollow-up device; a photographic lens; a drive means which drives saidphotographic lens; a focusing detection means which detects the focusadjustment condition of said object to be photographed for a pluralityof focus detection regions; a focus detection mode changeover meanswhich is operated to change over between a continuous mode in whichfocus adjustment movement of said photographic lens is repeatedlyperformed by said drive means for one or another of said plurality offocus detection regions based upon the output of said focusing detectionmeans, and a multi mode in which focus detection is possibleindependently for said plurality of focus detection regions, based uponthe output of said focusing detection means; and a control means whichchanges over from said continuous mode to said multi mode when saidtarget follow-up device is started by said follow-up starting means,even if said continuous mode is selected by said focus detection modechangeover means.

Further, according to the present invention as expressed in anothermanner, there is proposed a camera having a target follow-up devicewhich follows up the movement of an object to be photographed,comprising: a follow-up starting means which starts said targetfollow-up device; a photographic lens; a drive means which drives saidphotographic lens; a focusing detection means which detects the focusadjustment condition of said object to be photographed for a pluralityof focus detection regions; a focus detection mode changeover meanswhich is operated to change over between a one shot mode in which, whenonce the focus adjustment condition of said photographic lens has beendetected, based upon the output of said focusing detection means for oneor another of said plurality of focus detection regions, subsequentlythe movement of said photographic lens by said drive means iscompulsorily stopped, a continuous mode in which focus adjustmentmovement of said photographic lens is repeatedly performed by said drivemeans for one or another of said plurality of focus detection regionsbased upon the output of said focusing detection means, and a multi modein which focus detection is possible independently for said plurality offocus detection regions, based upon the output of said focusingdetection means; and a control means which, when said target follow-updevice is started by said follow-up starting means, changes over fromsaid one shot mode or said continuous mode to said multi mode, even ifsaid one shot mode or said continuous mode is selected by said focusdetection mode changeover means.

Further, according to the present invention as expressed in anothermanner, there is proposed a camera having a zoom lens, comprising: alight receiving means which comprises a plurality of light receivingelements disposed in the form of a matrix; an extraction means whichextracts a characteristic signal of an object to be photographed basedupon the output of said light receiving means; an operating means whichis operated in order to input a fixed magnification; a calculating meanswhich calculates the proportion which said characteristic signal of saidobject to be photographed, extracted by said extraction means, occupiesof the photographic field; and a zoom control means which drive controlssaid zoom lens so as to bring said proportion calculated by saidcalculating means substantially to correspond to said fixedmagnification.

Further, according to the present invention as expressed in anothermanner, there is proposed a camera having a zoom lens, comprising: alight receiving means which comprises a plurality of light receivingelements disposed in the form of a matrix; an extraction means whichextracts a characteristic signal of an object to be photographed basedupon the output of said light receiving means; a memory means whichrecords said characteristic signal of said object to be photographed,extracted by said extraction means; a calculating means which calculatesthe proportion said object to be photographed occupies of thephotographic field, based upon said characteristic signal of said objectto be photographed recorded by said memory means; and a zoom controlmeans which drive controls said zoom lens so as to maintain saidproportion calculated by said calculating means at a substantiallyconstant proportion.

Further, according to the present invention as expressed in anothermanner, there is proposed a camera having a zoom lens, comprising: alight receiving means which comprises a plurality of light receivingelements disposed in the form of a matrix; an extraction means whichextracts a characteristic signal of an object to be photographed basedupon the output of said light receiving means; a memory means whichrecords said characteristic signal of said object to be photographed,extracted by said extraction means; an operating means which is operatedin order to record in said memory means a characteristic signal for anobject to be photographed which is present within a predetermined regionof the photographic field; a calculating means which, when saidoperating means is operated, calculates the proportion which said objectto be photographed occupies of the photographic field, based upon saidcharacteristic signal recorded by said memory means; and a zoom controlmeans which drive controls said zoom lens so as to maintain saidproportion calculated by said calculating means at a substantiallyconstant proportion.

Further, according to the present invention as expressed in anothermanner, there is proposed a camera having a target follow-up devicewhich follows up the movement of an object to be photographed,comprising: a follow-up starting means which starts said targetfollow-up device; a photographic mode setting means which possesses aplurality of photographic modes for performing suitable exposure controlaccording to photographic circumstances, and which includes among saidplurality of photographic modes a moving target photographic mode whichis suitable for when said object to be photographed is moving; and acontrol means which controls said photographic mode setting means so asto set said moving target photographic mode, when said target follow-updevice is started by said follow-up starting means.

According to the present invention, because the time period required forthe calculation is reduced, it is possible to perform follow-up actionto detect the position even of a quickly moving target rapidly andaccurately. Further, since the calculation region is set appropriately,or a predicted position of the target is calculated, therefore even ifthe position of the target varies widely it can be followed upaccurately and reliably. Yet further, it is possible adequately to copewith alteration of the shape or the brightness of the target.

Moreover, according to the present invention, since the target follow-updevice is started in accordance with the photographic mode, thereby itbecomes possible always to perform the most suitable type of controlaccording to the photographic situation of the object to bephotographed. Further, when a fixed magnification is selected, it ispossible always to maintain this fixed magnification based upon theproportion of the photographic field occupied by the principal object tobe photographed. Yet further, since the focus detection mode is set tothe most suitable mode when the target follow-up device is started,focus adjustment is performed properly and accurately.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic side view of a camera to which a target follow-updevice according to the first embodiment of the present invention hasbeen applied;

FIG. 2 is a figure showing details of an imaging sensor included in theFIG. 1 camera;

FIG. 3 is a figure showing the detailed arrangement of elements andsub-elements on the imaging sensor of FIG. 2;

FIG. 4A and FIG. 4B are flow charts for a main program which is executedby a calculation device incorporated in the FIG. 1 camera, in order torealize the operation of the first embodiment of the present invention;

FIG. 5 is a flow chart for a first version of a subroutine SR1 for colorselection, which is called from the FIG. 4A main program;

FIG. 6 is a flow chart for a second version of this color selectionsubroutine SR1;

FIG. 7 is a flow chart for a third version of this color selectionsubroutine SR1;

FIG. 8 is a flow chart showing the details of a subroutine SR2 forminimum residue calculation, which is called from the FIG. 4B mainprogram;

FIG. 9 is a flow chart showing the details of a subroutine SR3 forsetting a calculation region C for the next iteration round the mainprogram loop, which is called from the FIG. 4B main program;

FIG. 10 is a figure showing the definition of a velocity vector, and theprediction from this velocity vector of the position of an object to bephotographed;

FIG. 11 is a figure showing the positional relationships of a follow-upregion B, the calculation region C, and a detection region D when theseregions are as initially set;

FIG. 12 is a figure showing the relationship between the indices for thecolor outputs from the follow-up region B and from the detection regionD, when these regions are as initially set before residue calculation;

FIG. 13 is a figure showing the relationship between the indices for thecolor outputs from the follow-up region B and from the detection regionD, during residue calculation;

FIG. 14 is a figure showing the relationship between the indices for thecolor outputs from the follow-up region B and from the detection regionD, when residue calculation has been completed;

FIG. 15 is a figure for explanation of the procedure for calculationregion setting;

FIG. 16 is a figure showing the positional relationships between thefollow-up region B and four neighboring regions used for the secondversion of the color selection subroutine SR1, whose flow chart is shownin FIG. 6;

FIG. 17 is a schematic side view of a camera incorporating a targetfollow-up device which is a second embodiment of the present invention;

FIG. 18 is a block diagram of the camera shown in FIG. 17;

FIG. 19 is a flow chart which describes the flow of a main routine of aprogram executed by a CPU 61 incorporated in the FIG. 17 camera;

FIG. 20A and FIG. 20B are flow charts for explaining a first version ofa subroutine S11 of the program executed by the CPU 61 for settingphotographic mode;

FIG. 21 is a flow chart for explaining a first version of a subroutineSR12 of the program executed by the CPU 61 for selecting a predeterminedmagnification ratio photographic mode;

FIG. 22 is a flow chart for explaining a second version of thispredetermined magnification ratio photographic mode selecting subroutineSR12;

FIG. 23 is a flow chart for explaining a first version of a subroutineSR13 of the program executed by the CPU 61 for selecting an AF mode;

FIG. 24 is a flow chart for explaining a second version of this AF modeselecting subroutine SR13;

FIG. 25 is a flow chart for explaining a third version of this AF modeselecting subroutine SR13;

FIG. 26 is a flow chart for explaining a second version of the settingphotographic mode; and:

FIG. 27A and FIG. 27B are flow charts for explaining a third version ofthe setting photographic mode.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS First Embodiment

FIG. 1 is a schematic side view showing a camera incorporating a targetfollow-up device according to the first embodiment of the presentinvention. Referring to this figure, the reference numeral 1 denotes asingle lens reflex camera (hereinafter simply termed a camera) suitablefor silver halide photography. In this camera, light rays emanating froman object to be photographed (which is located generally to the left ofthe figure but is not particularly shown) pass through a lens 2 providedin a front portion of the camera 1, are reflected from a mirror 3 in anupwards direction in the figure, and are focused to form an image on ascreen 4. The photographer inspects this image of the object to bephotographed which is focused on the screen 4 through a prism 5 and aneyepiece lens 9. The image of the object to be photographed focused onthe screen 4 is also focused again on an imaging sensor 7 by a lens 6,and the output from this imaging sensor 7 is processed by a calculationdevice 8 which may be implemented by a microcomputer or the like of aper se known type, not particularly shown. The calculation device 8 isprovided with a memory unit not particularly shown in the figure, and inthis memory unit a reference image to be used as a template is stored asdigital data.

FIG. 2 is a figure showing details of the imaging sensor 7. As shown inFIG. 2, the imaging sensor 7 comprises a number of imaging elements 10arranged in a rectangular matrix, shown exemplarily in the figure asbeing 18 horizontally by 12 vertically, and an image equivalent to thephotographic image field which is being inspected by the photographer isfocused on this imaging sensor 7 by the lens 6. In FIG. 2, A denotes avisual region (namely, typically, the entire imaging sensor 7)corresponding to the entire photographic field which is being inspectedby the photographer, while B denotes a follow-up region which shows thedetected position of the target. In the shown first embodiment of thepresent invention, this follow-up region B is a 4×4 rectangle of imagingelements. The manner in which this follow-up region B is set will bedescribed hereinafter. The photographer can check the initial positionfor the follow-up region B within the visual region A, and the positionduring target follow-up if the photographer desires by ocularinspection, for example by a liquid crystal means or the like oftransparent type provided on the screen 4.

Each of the imaging elements 10 is divided into three imagingsub-elements 11a through 11c, as shown in FIG. 3 which is a figureshowing the detailed arrangement of the imaging elements 10 and theirimaging sub-elements 11 on the imaging sensor 7. Further, as shown inthe figure, each of the imaging sub-elements 11a is fitted with a redfilter, each of the imaging sub-elements 11b is fitted with a greenfilter, and each of the imaging sub-elements 11c is fitted with a bluefilter. Accordingly an RGB color output signal corresponding to theimage of the object to be photographed is obtained from the imagingelements 10, each being considered as a unit. In the followingdescriptions, each of these colors red, green, or blue respectively willbe indicated by the capital letter R, G, or B prefixed to itscorresponding imaging element, signal, etc.

Next, the operation of the camera 1 according to the first embodiment ofthe present invention will be explained with reference to the flowcharts shown in FIGS. 4, 5, 8, and 9, and with reference to theexplanatory diagrams of FIGS. 10 through 15.

(1) The color selection process

The calculation device 8 starts to obey the program shown in the flowcharts of FIG. 4A and FIG. 4B, at the stage in which the object to bephotographed (which will be set as the target according to thephotographer observing its image) is captured within the follow-upregion B, by the photographer giving an order for target acquisition ofthe object to be photographed, for example by the operation of firststroke operation of a release button (not shown in the figure) or thelike. Moreover, when this program is started, the follow-up region B isset to be positioned exactly in the center of the visual region A, asshown in FIG. 2.

First, in the step S1, the RGB output signal from each imaging element10 of the imaging sensor 7 within the follow-up region B is read in. TheRGB output signals thus obtained from within the follow-up region B arestored within a temporary memory element (not particularly shown)incorporated in the calculation device 8. Next, in the subroutine SR1, acolor selection process is performed, in which one or the other of thecolor output signals (the R signal, the G signal, or the B signal) isselected from the RGB output signal thus obtained from within thefollow-up region B.

The details of a first variant of this subroutine SR1 for the colorselection process, which is employed in this first embodiment, are shownin the flow chart of FIG. 5. First, in the step S101, for each of thecolors R, G, and B, the corresponding portions of the RGB output signalsobtained in the step S1 are summed over all the imaging elements 10within the follow-up region B, so as to produce three total values forthe three colors. Next, in the step S102, the maximum value of thesethree total values obtained in the step S101 as a result of the triplesummation procedure is obtained, and that one of the colors R, G, and Bis designated as the selected color α, the summation for which producedthat maximum value. This FIG. 5 subroutine then terminates and returnscontrol to the main program whose flow chart is shown in FIG. 4A.

Next, in the step S2 of this main program, the output signals from theRGB output signals from the follow-up region B which correspond to thecolor α which was selected in the subroutine SR1 are set as B_(ij) (i,j=1 . . . 4), and are stored in a matrix form in a memory device (notparticularly shown) incorporated in the calculation device 8. Thesevalues B_(ij) are reference image data in which is included an image(the target) of the desired object to be photographed, the position ofwhich should be followed up.

(2) Setting the calculation region

In the next step S3, the output signal for the color α is obtained fromthe imaging sensor 7 for each imaging element 10 in the visual region A.This color α output over the visual region A is stored in the temporarymemory means, not particularly shown, of the calculation device 8.

In the next step S4, the velocity (taken as a vector quantity) and theacceleration (taken as a vector quantity also) of the follow-up region Bare calculated. As shown in FIG. 10, the follow-up region B is moved inthe visual region A along with the movement of the object to bephotographed, and in the step S9 to be described hereinafter itsposition is set each time the position of the object to be photographedis detected. The velocity vector V (v_(x),v_(y)) is defined as thenumber of imaging elements over which the follow-up region B has movedin the horizontal direction (also in the following called the Xdirection) and in the vertical direction (also in the following calledthe Y direction), during the time interval in which the detectionprocess in the step S9 for the target object to be photographed isexecuted. Further, the acceleration vector AC (a_(x),a_(y)) is definedas the amount of change (a_(x) =v_(1x) -v_(2x), a_(y) =v_(1y) -v_(2y))in the X direction and in the Y direction between two neighboringvelocity vectors V₁ and V₂ (refer to FIG. 10).

In the next decision step S5, a decision is made as to whether or noteach individual one of the absolute values |V_(1x) -V_(2x) |, |V_(1y)-V_(2y) | of the amount of change of each component of the velocityvector (i.e., of each component of the acceleration vector) calculatedin the previous step S4 is. less than a previously determined thresholdvalue T₁ ; and if the result of this decision is YES then the flow ofcontrol proceeds to the step S6, while if the result of the decision isNO then the flow of control proceeds to the step S7. This thresholdvalue T₁ is for determining whether or not the object to be photographedis moving at an almost constant velocity, and should be suitably set inview of inaccuracies etc. in measurement of the velocity vector.

In the step S6, because the amount of change between the velocityvectors V₁ and V₂ is a small value less than the threshold value T₁, itis judged that the object to be photographed will also move with almostthe same velocity vector during the next time interval for the nextiteration of this program loop, and accordingly, supposing V₀ =V₁, asshown in FIG. 10, the calculation region C (which will be describedlater) is set to a position displaced just by the velocity vector V₀from the present position of the follow-up region B (i.e. the positionof the follow-up region B which was set according to follow-up actionlast time round this program cycle). After this, the flow of controlproceeds next to the subroutine step SR2.

In the step S7, a decision is made as to whether or not the absolutevalue of the acceleration vector AC calculated in the step S4 is greaterthan a previously determined threshold value T₂, and if the result ofthis decision is YES then the flow of control proceeds to the step S8,while if the result of the decision is NO then the flow of controlproceeds to the subroutine step SR2. This threshold value T₂ is fordetermining whether or not the object to be photographed is moving at avelocity which is changing abruptly, and should be suitably set in viewof the size of the calculation region, etc.

In the step S8, since the value of the acceleration vector AC is a largevalue greater than the threshold value T₂, it is judged that themovement of the object to be photographed is irregular (its speed mayperhaps get greater, or it may perhaps stop unexpectedly, etc. ), andaccordingly, supposing V₀ =V₁, a calculation region C larger than thenormal calculation region (for example, twice as large in both the Xdirection and in the Y direction, or even more) is set at a positiondisplaced just by the velocity vector V₀ from the present position ofthe follow-up region B.

In a case where both of the decisions at step S5 and S7 are negated, acalculation region C is set to a position same as a position of acalculation region which was set last time round this program cycle.

(3) Minimum residue calculation

Minimum residue calculation is performed in the next subroutine stepSR2. The details of the subroutine SR2 for minimum residue calculationare shown in the flow chart of FIG. 8. First, in the step 201,initialization of various values used in the calculation is performed.In practice, this includes setting a detection region D to be positionedat the upper left corner of the calculation region C as shown in FIG.11, and setting a counter n to 1.

FIG. 11 is a figure showing the positional relationships of thefollow-up region B, the calculation region C, and the detection region Dwhen they are initially set. Referring to this figure, C denotes thecalculation region, which is the region in which the object to bephotographed is searched for. In the figure it is shown by way ofexample that the calculation region C is set to a region which iscentered on the follow-up region B and which is wider than the follow-upregion B by just one imaging element both leftwards and rightwards inthe X direction and upwards and downwards in the Y direction; but aswill be described hereinafter the size of the calculation region C isnot to be considered as being limited to the example shown in thisfigure. D denotes the detection region, which is the region in whichresidue calculation with respect to the follow-up region B is performed,and in the shown example, just like the follow-up region B, thisdetection region D is a 4×4 rectangle of imaging elements. Next, in thestep S202, the output for the color α from each imaging element in thecalculation region C is obtained.

Next, in the step S203, the output D_(ij) (i,j =1 . . . 4) for the colorα from each element of the detection region D set in the step S201 isobtained from the outputs for the entire visual region A which werestored in step S3 of the main routine of FIG. 4A. The relationshipbetween the indices for the outputs B_(ij) for the color α for thefollow-up region B (these are also reference image data) and the indicesfor the outputs D_(ij) for the color α for the detection region D isshown in FIG. 12, for the case when these regions are initially set asshown in FIG. 11.

Next, in the step S204, the residue is calculated for the referenceimage data B_(ij) and the outputs D_(ij) for the detection region D.This residue is defined as being the sum over all possible combinationsof the indices i and j of the absolute values of the differences betweencorresponding ones of these quantities, i.e. between ones of thesequantities which have the same indices; in other words, in the showncase, this residue is equal to ΣΣW_(ij) |B_(ij) -D_(ij) |, where thesummation is over i=1 to 4 and j=1 to 4. The coefficients W_(ij) areweighting coefficients, and this coefficients prevent the residue frombecoming minimum at points where the object to be photographed does notexist (so called pseudo-watching). This phenomenon may because theamount of information reduce when calculating the absolute values of thedifferences, and in addition, the amount of information reducefurthermore when calculating the total sum. These weighting coefficientsare set to be the greater the closer are the imaging elements to thecenter, in consideration of the fact that the closer an imaging elementis to the center of the reference image data B_(ij) the higher is theprobability that it gives important data about the object to bephotographed; and their values are obtained by experiment. One effectivesetting for these weighting coefficients W_(ij) is for W_(ij) to beequal to 2 if both i and j are either 2 or 3, and for W_(ij) to be equalto 1 in all other cases. The residue is set as a variable Sum_(n) wheren is a suffix indicating the number of times calculation has beenrepeated within the calculation region C, and it is stored in the memorymeans, not particularly shown, of the calculation device 8.

Next, in the step S205, the detection region D is moved for the nextresidue calculation, and the value of the counter n is incremented byone. In detail, the detection region D is moved by one imaging elementto the right if this can be done within the calculation region C; but,if the right edge of the detection region D is already in coincidencewith the right edge of the calculation region C, then the detectionregion D is moved down by one imaging element and is also shifted allthe way to the left within the calculation region C, so that the leftedge of the detection region D comes into coincidence with the left edgeof the calculation region C. By doing this, the detection region D isrepeatedly moved from left to right as seen in FIGS. 11 and 12, eachtime also being moved downwards by one row of detection elements. FIG.13 shows the relationship between the indices for the color α outputB_(ij) from the follow-up region B and for the color α output D_(ij)from the detection region D, in the state where the detection region Dhas been shifted by one detection element to the right from its FIG. 12position.

Next, in the step S206, a decision is made as to whether or not thevalue of the counter n has yet reached its final value (in the shownexample this final value is 10, because there are nine possiblepositions for the detection region D with respect to the follow-upregion B), and if the result of this decision is YES then the flow ofcontrol proceeds to the step S207, while if the result of the decisionis NO then the flow of control returns to the step S203 and the processdescribed above is repeated. By doing this, residue calculation withrespect to the reference image data is repeated while moving thedetection region D, and 9 residues are calculated. FIG. 14 is a figureshowing the relationship between the indices for the color α outputsB_(ij) from the follow-up region B and the indices for the color αoutputs D_(ij) from the detection region D, when residue calculation hasbeen completed.

Next, in the step S207, over n=1 to 9, it is detected which value of ngives the minimum value of Sum_(n),i.e. the minimum residue. In the nextstep S208, it is decided that the object to be photographed is presentin the detection region D which gives the minimum value of Sum_(n), andthe position of this detection region D is calculated. This FIG. 8subroutine then terminates and returns control to the main program whoseflow chart is shown in FIG. 4B.

(4) Moving the follow-up region and setting the next calculation regionfor the next iteration

In the next step S9 of the FIG. 4B main routine, the follow-up region Bis moved to the position of the detection region D which includes theobject to be photographed calculated in the subroutine SR2 describedabove. In this way, a moving object to be photographed is followed up.Further, in this first embodiment, the positions of the follow-up regionB for the last two iterations of the main program loop are preservedwithin the memory, because as shown in FIG. 10 past positions of thefollow-up region B are necessary for the calculation of the velocityvector V and of the acceleration vector AC.

Next, in the subroutine step SR3, the calculation region C for the nextiteration of the main program loop is set. If, as shown in FIG. 15, thewidth (expressed in terms of number of imaging elements) in thehorizontal direction of the border that appears between the calculationregion C and the follow-up region B when they are centered together istermed Sx, and the width in the vertical direction of this border istermed Sy, then the following relationship holds between the number n ofrepetitions of the residue calculation process and Sx and Sy:

    n=(2S.sub.x +1)×(2S.sub.y +1)

and, if the calculation region C is made larger so as not to lose sightof the object to be photographed, then the number n of residuecalculation process repetitions is increased, and a longer time periodis required for the calculation of the position of the object to bephotographed, and the paradoxical circumstance arises that as a result,after all, the object to be photographed can be lost sight of. Thus thesize of the calculation region C is appropriately set in the subroutinestep SR3, based upon the minimum residue which was calculated in theprevious subroutine SR2.

The details of this subroutine SR3 for setting the calculation region Cfor the next iteration around the main program loop are shown in theflow chart of FIG. 9. First, in the decision step 301, a decision ismade as to whether or not the minimum residue obtained in the subroutineSR2 is greater than a threshold value T₃, and if the result of thisdecision is YES then the flow of control proceeds to the step S302,while if the result of the decision is NO then the flow of controlproceeds to the step S305.

In the steps S302 and S303, since the minimum residue is a value greaterthan the threshold value T₃, it is deemed that the movement activity ofthe object to be photographed is fairly high, and the process isperformed of enlarging the calculation region C. In detail, thehorizontal border width Sx and the vertical border width Sy (bothexpressed in terms of number of imaging elements) for the calculationregion C are both set to the value (minimum residue/T₃ +constant). Sincein the decision step S301 it was decided that the minimum residue isgreater than T₃, the value of the expression (minimum residue/T₃) isgreater than 1, and is greater, the greater is the minimum residue.Next, in the step S304, in order to stabilize the size of the enlargedcalculation region C, the threshold value T₃ is increased by a constantvalue. After this step, the FIG. 9 subroutine then terminates andreturns control to the main program whose flow chart is shown in FIG.4B.

On the other hand, in the steps S305 and S306, since the minimum residueis a value equal or less than the threshold value T₃, it is deemed thatthe movement activity of the object to be photographed is fairly low,and the process is performed of shrinking the calculation region C byreducing the horizontal border width Sx and the vertical border width Syby one imaging element each. And next, in the step S307, the thresholdvalue T₃ is decreased by a constant value, so as to provide for thefurther movement activity of the object to be photographed. After thisstep, the FIG. 9 subroutine again terminates and returns control to themain program.

The horizontal border width Sx and the vertical border width Sy set inthe steps S302 and S303, or in the steps S305 and S306, are used forsetting the calculation region C in the steps S6 and S8 of the mainprogram of FIG. 4B.

(5) Refreshing the reference image

In the step S10 of the main program, a decision is made as to whether ornot the minimum residue which was obtained in the subroutine SR2 isgreater than a threshold value T₃, and if the result of this decision isYES then the flow of control loops back to the step S3 to perform theprocess described above again. On the other hand, if the result of thedecision is NO, then the flow of control proceeds to the step S11.

In this step S11, the procedure of refreshing the reference image, i.e.of renewing it, is performed. This reference image renewal procedure isperformed in order to correspond to continuous change in the object tobe photographed, for instance the change from a full frontal view of aface to a profile view. In practice, while the input image data is beinginputted, some proportion of the reference image data is renewedincluding new data. In this step, values given by the followingequation:

    B.sub.ij =(1-k)B.sub.ij +kD.sub.ij (i,j=1 . . . 4)

are substituted for the reference image data B_(ij). Here, k should bebetween 0 and 1 inclusive, and values from about 0.1 to about 0.2 havebeen found to give good results upon trial. The D_(ij) are the datarelating to the detection region D in the subroutine SR2 for which thevalue Sum_(n) took the minimum value. However, when the movement of theobject to be photographed is chaotic, or in the case that an obstaclepasses in front of the object to be photographed, since data notrelating to the object to be photographed would undesirably be input ifthe process of refreshing the reference image were to be performed inthese cases, therefore in the step S10 described above the minimumdeviation is checked and is found to be larger than the threshold valueT₃, and accordingly refreshment of the reference image is prohibited.Finally, the flow of control returns to the step S3 and the abovedescribed process is repeated.

According to the operation described above, it is possible to follow upthe position in the viewfinder screen of the object which is to bephotographed by the camera 1. It should be understood that, since in theshown first embodiment the output for the one color α is selected fromthe RGB output signal from the imaging sensor 7 for calculation of theresidue, thereby the time period required for the calculations fordetection of the object to be photographed can as a whole be madeshorter by comparison to the case in which residue calculation isperformed for the entire RGB output signal, and in this manner it ispossible rapidly to follow up the position of even a quickly movingobject to be photographed, such as the point of attention in a sportsscene or the like. Furthermore, since the color α which is selected isthat color for which the maximum output is obtained, and since theresidue calculation is performed using the output signal for that colorα, thereby it is possible to choose the most distinctive one of thecolors for detection of the object to be photographed, and by thisselection of the most appropriate color output signal diminution of theamount of information is compensated for and accurate position detectionis made possible.

Further, since the reference image data is partially refreshed when theposition of the target is detected, thereby even if the form of theobject to be photographed changes continuously or if brightness of theobject to be photographed changes, it is possible accurately andreliable to capture the object to be photographed, and accordinglyaccurate position detection is made possible.

Moreover, in the shown first embodiment of the present invention, thecalculation region C is provided within the visual region A of theimaging sensor 7, the speed and the acceleration of the object to bephotographed are measured, and the next position of the calculationregion C is predicted. In addition, the size of the calculation region Cis appropriately controlled according to the magnitude of the minimumresidue. By doing this, it is possible to prevent the situation in whichthe object to be photographed is lost sight of, as was the problem withthe examples of the prior art which were discussed above, and stable,reliable and accurate following up of the object to be photographed ismade possible. Accordingly, the fact that it becomes unnecessary toobtain the residue for the entire screen is also very helpful, and it ispossible rapidly to follow up the position of even a quickly movingobject to be photographed, such as the point of attention in a sportsscene or the like.

Variants of First Embodiment

The manner in which the color selection process is performed is notlimited to the one shown and discussed above with reference to FIG. 5,in which the color α which provides the maximum output is selected;other possibilities are also available.

FIG. 6 is a flow chart for a second version of the color selectionsubroutine SR1. In the first step S401, the RGB outputs of each imagingelement obtained in the step S1 for the follow-up region B are summedover each of the three colors R, G, and B. Next, in the step S402, theRGB outputs for each imaging element are likewise summed for each of thethree colors R, G, and B over four neighboring regions B1 through B4,which are defined as shown in FIG. 16 which shows the positionalrelationships between the follow-up region B and these four neighboringregions B1 through B4, each being a 4×4 rectangle of imaging elementsjust like the follow-up region B, and neighboring the follow-up region Bon its upper side, on its lower side, on its right side, and on its leftside respectively. And, finally, in the step S403, for each of thecolors R, G, and B, the differences between the total in the follow-upregion B obtained in the step S401 and the totals in each of the fourneighboring regions B1 through B4 obtained in the step S402 arecalculated and the absolute values of these differences are addedtogether, and that color is selected which produces Z5 the maximumresult. This second FIG. 6 version of the subroutine SR1 then terminatesand returns control to the main program whose flow chart is shown inFIG. 4A.

FIG. 7 is a flow chart for a third version of the color selectionsubroutine SR1. In the first step S501, first, eight detection regionsare defined, each being a 4×4 rectangle of imaging elements just likethe follow-up region B, and each being displaced from the follow-upregion B by one imaging element, respectively upwards, downwards,rightwards, leftwards, diagonally upwards and rightwards, diagonallydownwards and rightwards, diagonally upwards and leftwards, anddiagonally downwards and leftwards. And then the RGB outputs of eachimaging element for these eight detection regions and for the follow-upregion B are summed over each of the three colors R, G, and B, and thedifferences (the residues)between the total in the follow-up region Band the totals in each of the eight detection regions are calculated foreach of the three colors R, G, and B. Next, in the step S502, for eachcolor, it is determined which one of the eight residues has the minimumvalue. Finally, in the step S503, that color (R, G, or B) is selected,for which the minimum value residue determined in the step S502 is thegreatest. This third FIG. 7 version of the subroutine SR1 thenterminates and returns control to the main program whose flow chart isshown in FIG. 4A.

Now, it is not essential to the present invention for the referenceimage data used as a template to be sampled before following up of theobject as was the case with the shown first embodiment; provided that itis possible to known the form etc. of the target in advance, it is alsoacceptable for reference image data relating to the target to be storedin the memory beforehand. In the same manner, it is not essential to thepresent invention for the input image data to be input in a real timefashion; it is also acceptable for it to be previously recorded data.

Further, although in the shown first embodiment the setting of thecalculation region was performed using both the velocity vector and theacceleration vector, it would also be acceptable to use only the one orthe other of these. In the case of using the acceleration vector, sinceif the acceleration vector is zero this means that the target is movingat a constant velocity, therefore if the acceleration and the initialvelocity are detected it is possible to set both the position and thesize of the calculation region appropriately.

Second Embodiment

FIG. 17 is a schematic side view of a camera incorporating a targetfollow-up device which is a second embodiment of the present invention.

Referring to this figure, a portion of the light that passes through aphotographic lens 51 also passes through a main mirror 52 and isreflected off a sub-mirror 53 onto a focus detection device 54, whilethe remainder of the light is reflected off the main mirror 52, passesthrough a focus screen 55, is deflected by passing through a pentaprism56, and then enters the eye (not shown) of the user of the camera via aneyepiece lens 57. Further, a portion of this light which has passedthrough the pentaprism 56 is deflected by a half silvered mirror 57a, isrefocused by a lens 58, and falls on a light receiving unit 59 which isfor detecting the object to be photographed. The reference symbol 60denotes a shutter device.

The photographic lens 51 comprises a focusing lens group 51a and azooming lens group 51b. The focusing lens group 51a is axially driven bya motor M2 according to a focus detection signal which is calculated andoutput by a CPU 61, based upon the output signal from the focusdetection device 54. Further, the zooming lens group 51b is axiallydriven by a motor M1 via the CPU 61, based upon a signal from a controlswitch described hereinafter which is for setting the magnification forphotography; and focal length to which the combination of the focusinglens group 51a and the zooming lens group 51b is focused is variedaccording to this driving of the zooming lens group 51b.

FIG. 18 is a block diagram of the camera shown in FIG. 17. As shown inthis figure, the light receiving unit 59 for detecting the object to bephotographed comprises a light receiving device 59a which is made upfrom (for example) several hundred imaging elements, and a processingcircuit 59b which processes the output signals from these imagingelements of the light receiving device 59a and outputs them to the CPU61. Color filters of the three different colors R (red), G (green), andB (blue) are fitted over the imaging elements of the light receivingdevice 59a, and thereby an RGB output signal corresponding to the objectto be photographed is produced.

As shown in FIG. 18, the focus detection device 54 comprises a focusdetection element 54a which can detect focus position for a plurality ofareas (the focus detection regions) in the photographic field of view(in the figure there are shown, by way of example, five areas for whichthe focus position can be detected), and a processing circuit 54b whichperforms A/D conversion processing etc. on the output signal of thisfocus detection element 54a and outputs a signal to the CPU 61.

The reference symbol 70 denotes a photographic mode selection unit,which can be used by the camera user to set the mode for performingphotography--for example, either a sports photography mode for takingsports photographs in which the most suitable exposure control methodfor performing sports photography is used, or a portrait photographymode in which the most suitable exposure control method for performingportrait photography is used. This photographic mode selection unit 70comprises a plurality of control switches provided on the outer casingof the camera for the various photographic modes (in the figure, only asports photography mode switch SW70a and a portrait photography modeswitch SW70b are exemplarily indicated), and circuitry not particularlyshown in the figure. The appropriate control switch is operatedaccording to the photographic mode desired, and a signal indicating thephotographic mode thereby selected is sent from the photographic modeselection unit 70 to the CPU 61, so as to set the desired photographicmode.

The reference symbol 71 denotes a focus detection mode changeover unit,which can be used by the camera user to set the mode for focusdetection--for example, either a one shot mode (S-AF mode) in whichbased upon the output from the focus detection device 54 once afterdetecting the condition in which the photographic lens 51 is on focus,the movement of the photographic lens 51 is stopped, or a continuousmode (C-AF mode) in which based upon the output from the focus detectiondevice 54 the focus adjustment operation of the focusing lens group 51ais repeatedly performed, or a multi mode (M-AF mode) in which based uponthe output from the focus detection device 54 focus detection can beperformed independently for each of a plurality of areas in thephotographic field of view. In more detail, in this second embodiment ofthe present invention, this multi or M-AF mode is a mode in which focusdetection is performed independently for each of a plurality of areas inthe photographic field of view, and the focus adjustment operation isthen performed for the most suitable one of this plurality of focusdetection areas. Exemplarily, that one of the plurality of focusdetection areas which is determined to be at the closest photographicdistance may serve as the most suitable one. On the other hand, in theone shot or S-AF mode or the continuous or C-AF mode, the user of thecamera chooses one area from the plurality of focus detection areasprovided on the focus detection device 54, and the focus adjustmentoperation is then performed for this selected focus detection area. Thisfocus detection mode changeover unit 71 comprises a plurality of controlswitches provided on the outer casing of the camera for the variousfocus detection modes (in the figure, only a S-AF mode switch SW71a, aC-AF mode switch SW71b, and a M-AF mode switch SW71c are exemplarilyindicated), and circuitry not particularly shown in the figure. Theappropriate control switch is operated according to the focus detectionmode desired, and a signal indicating the focus detection mode therebyselected is sent from the focus detection mode changeover unit 71 to theCPU 61, so as to set the desired focus detection mode. In the followingdescription, the focus detection mode will in the interests of brevityalso be referred to as the AF mode.

The reference symbol 72 denotes a predetermined magnificationphotographic mode setting unit which can be used by the camera user toset the so called auto zoom mode, i.e. the mode for automaticallydriving the zooming lens group 51b according to change in thephotographic distance so as to ensure that the photographicmagnification always maintains a value which is determined in advance.This predetermined magnification photographic mode setting unit 72comprises a plurality of control switches provided on the outer casingof the camera for the various possible preset photographicmagnifications available (in the figure, only a first magnificationswitch SW72a and a second magnification switch SW72c are exemplarilyindicated). The appropriate control switch is operated according to thephotographic magnification desired, and a signal indicating thephotographic magnification thereby selected is sent from thepredetermined magnification photographic mode setting unit 72 to the CPU61, so as to set the desired photographic magnification.

The reference symbol 73 denotes a follow-up area designation member 73,which is constructed so as to operate together with the photographicmode selection unit 70, the focus detection mode changeover unit 71, andthe predetermined magnification photographic mode setting unit 72described above. This follow-up area designation member 73, as shown inFIG. 18, normally detects characteristic information (a color differencesignal, a luminance signal, or the like) relating to the object to bephotographed in the follow-up area 55b set in the central portion of thephotographic field of view 55a and outputs such information to the CPU61, and further instructs the CPU 61 to follow up the object to bephotographed which possesses such detected characteristic information tobe followed up in the photographic field of view 55a. Such a colordifference signal or luminance signal may be defined, for example,according to the NTSC (National Television Systems Committee) format,which is one of several standard television formats. For reference,according to the NTSC format, the following relationships hold betweenthe color difference signals and luminance signals and the RGB outputsignals: ##EQU1##

Further, the follow-up area designation member 73 is so constructed asto make it possible for the user of the camera to manually indicate theobject to be photographed which is to be followed up. That is to say,the follow-up area designation member 73 can freely set the position ofthe follow-up area 55b anywhere within the photographic field of view55a, and for example may consist of a pointing device such as atrackball or the like by which the follow-up area 55b can be moved. Thisfollow-up area designation member 73 extracts the characteristicinformation relating to the object to be photographed within thefollow-up area 55b, based upon the output of the light receiving unit 59for discriminating the object to be photographed, and outputs thisinformation to the CPU 61. Furthermore, it is so arranged that theposition of the follow-up area 55b within the photographic field of view55a can be visually checked by the user of the camera, for example bythe use of a liquid crystal means of transparent type provided on thefocus screen 55.

The CPU 61 comprises a microcomputer, a memory device, and otherassociated elements of a per se conventional type, not particularlyshown, and starts follow-up action for the object to be photographedaccording to orders received from the follow-up area designation member73. Further, since the characteristic information (such as a colordifference signal) relating to the object to be photographed within thefollow-up area 55b is extracted by the follow-up area designation member73, the CPU 61 records in its memory device the color difference signal(or the like) from the follow-up area 55b which is designated on thescreen, and, by comparing this recorded color difference signal and thecolor difference signal around the perimeter of the designated follow-uparea 55b, detects what position on the photographic field of view 55athe principal object to be photographed has moved to.

In the following, the control operation of the camera according to thissecond embodiment will be explained with reference to the flow chartsshown in FIGS. 19 through 25.

(1) The main routine

The flow chart of FIG. 19 describes the flow of a main routine of theprogram executed by the CPU 61. The execution of this main routine ofFIG. 19 is started, for example, at the time point when a main switch(not particularly shown) of the FIG. 17 camera is operated to switch thepower source of the camera on.

First, in the step S21, various initialization settings are performedfor various control variables etc. of the program and for variousfunctions of the camera. Then, in the next subroutine step SR11, thedesired photographic mode is selected by the user of the camera via thephotographic mode selection unit 70, and exposure calculation isperformed according to this selected photographic mode. This subroutineSR11 will be described in detail hereinafter with reference to the flowcharts of FIG. 20A and FIG. 20B.

Next, in the subroutine SR12, the desired photographic magnification isselected by the user of the camera via the predetermined magnificationphotographic mode setting unit 72, and various control functions areperformed according to this selected photographic magnification. Thissubroutine SR12 will be described in detail hereinafter with referenceto the flow chart of FIG. 21.

Next, in the subroutine SR13, the desired focus detection mode isselected by the user of the camera via the focus detection modechangeover unit 71, and various control functions are performedaccording to this selected focus detection mode. This subroutine SR13will be described in detail hereinafter with reference to the flow chartof FIG. 22.

Next, in the decision step S22, a decision is made as to whether or notthe release button of the camera (not particularly shown) is beingoperated, and if the result of this decision is YES then it is judgedthat a shutter release order is being issued and the flow of controlproceeds to the step S23 and photography is performed according to a perse known exposure control sequence, while on the other hand if theresult of the decision is NO then the flow of control loops back toreturn to the step S21.

(2) Setting the photographic mode

Two versions of the details of the subroutine SR11 called from the flowchart of FIG. 19 are shown in the flow charts of FIG. 20A and FIG. 20B,and FIG. 27A and FIG. 27B. These flow charts will be used for explainingthe flow of control operation in the case that the most suitableexposure control mode is set for when the photographic mode is thesports photography mode or the like for when the object to bephotographed is a moving object.

(a) A first example of control

FIG. 20A and FIG. 20B are flow charts for showing a first example ofcontrol for setting the photographic mode.

First, in the step S601, a decision is made as to whether or not aphotographic mode suitable for photography of an object which is moving,such as the sports photography mode or the like, is being commanded bythe operation of the control switches of the photographic mode selectionunit 70; and, if such a photographic mode suitable for photography of anobject which is moving, such as the sports photography mode or the like,is being commanded, then the flow of control proceeds to the step S602;while, if a photographic mode suitable for photography of an objectwhich is stationary, such as the portrait photography mode or the like,is being commanded, then the flow of control proceeds to the step S608.

In the step S602, the CPU 61 sets the photographic mode according to theinput signal from the photographic mode selection unit 70 which iscommanding the sports photography mode or the like, and also sets thetarget object follow-up operational mode. Next, in the step S603, thefollow-up area designation member 73 picks out the characteristicinformation for the object to be photographed which is positioned withinthe follow-up area 55b which is itself set at a predetermined positionwithin the photographic field of view 55a, and the CPU 61 records thischaracteristic information, i.e. the color difference signal, in itsmemory.

Next, in the step S604, the color difference signal recorded in thememory of the CPU 61 and the color difference signal around theperimeter of the designated follow-up area 55b are compared, and it isdetermined what position on the photographic field of view 55a theprincipal object to be photographed has moved to. As for the details ofthis detection procedure, only an abbreviated description will be givenin this specification, since they are disclosed in (for example) U.S.Pat. No. 5,031,049. In summary, taking the R-Y color difference signaland the B-Y color difference signal to be respectively the vertical axisand the horizontal axis, the coordinates of the color difference signalsfor each imaging element are plotted on a plane, and by moving thecoordinates the direction of movement of the principal object to bephotographed is determined.

Next, in the step S605, photometric calculation processing is performedfor the photographic field of view 55a which puts the weight on themeasured light values of the region in which the principal object to bephotographed is present. Since this type of photometric processing isper se known in the art, its details will not be explained in thisspecification. Next, in the step S606, an exposure control value iscalculated based upon the result of this photometric calculationprocessing, and after this the subroutine SR11 then terminates andreturns control to the main program whose flow chart is shown in FIG.19.

On the other hand, if the flow of control is directed to the step S608,then, since it has been decided in the decision step S601 that thesports photography mode or the like is not being commanded, thereforethe CPU 61 sets the photographic mode to the portrait photography modeor the like, according to the input signal from the photographic modeselection unit 70.

Next, in the step S609, the light from the object to be photographedwhich is positioned in the central portion of the photographic field ofview 55a is received by the light receiving unit 59 for discriminatingthe object to be photographed. In a photographic mode suitable forphotography of an object which is stationary, normally, since it oftenhappens that the principal object to be photographed is located in thecentral portion of the photographic field of view 55a, therefore theform of the object to be photographed which exists in the centralportion of the photographic field of view 55a is discriminated, and itis beneficial to perform photometry by placing weight on thisdiscriminated region. Next, in the step S610, photometric calculationprocessing is performed which puts the weight upon the measured lightvalues of the central portion of the photographic field of view 55a,i.e. upon the region in which the principal object to be photographed isdecided to be present. Next, in the step S611, an exposure control valueis calculated based upon the result of this photometric calculationprocessing, and after this the subroutine SR11 then terminates andreturns control to the main program whose flow chart is shown in FIG.19.

(b) A second example of control

It is also possible to change the above described steps S602 throughS606 as will now be described. To wit, in the second example of controlshown in the flow chart of FIG. 27A, instead of the steps S602 throughS606, the steps S620 through S623 are substituted, and except for thisalteration the control process for this FIG. 27A and FIG. 27B flowcharts is the same as for the FIG. 20A and FIG. 20B flow charts.

In the step S620, the CPU 61 sets the photographic mode according to theinput signal from the photographic mode selection unit 70 which iscommanding the sports photography mode or the like, and also sets thetarget object follow-up operational mode. Further, the CPU 61, if thefocus detection mode is set to the one shot mode (S-AF mode), changesthe focus detection mode to the continuous mode (C-AF mode) or the multimode (M-AF mode).

To describe this process in more detail, if the camera is provided withboth of the two focus detection modes--the continuous mode (C-AF mode)and the multi mode (M-AF mode)--then a changeover is performed from thecontinuous mode (C-AF mode) to the multi mode (M-AF mode). Further, ifthe camera is provided with all three of the focus detection modes--theone shot mode (S-AF mode), the continuous mode (C-AF mode), and themulti mode (M-AF mode)--then a changeover is performed from either theone shot mode (S-AF mode) or the continuous mode (C-AF mode) to themulti mode (M-AF mode).

Next, in the step S621, a process of follow-up control for the object tobe photographed identical to that performed previously in the step S604is performed. And next, in the step S622, that focus detection region isselected which is nearest to the region in which the object to bephotographed which has been followed up exists (the area is selectedfrom the five areas of the focus detection element 54a for which focusdetection is possible), and focus adjustment control is performed so asalways to follow up the object to be photographed.

Next, in the step S623, an exposure control value is calculated in afashion identical to that performed previously in the steps S605 andS606, and after this the subroutine SR11 then terminates and returnscontrol to the main program whose flow chart is shown in FIG. 19.

(3) Setting the predetermined magnification photographic mode

The flow charts of FIGS. 21 and 22 serve for explaining two differentversions of the subroutine SR12 of FIG. 19. The flow charts of FIGS. 21and 22 are for explaining the flow of control when a predeterminedmagnification is set by the predetermined magnification photographicmode setting unit 72. When a predetermined magnification photographicmode is set, zooming action is performed while maintaining a previouslydetermined photographic magnification.

(a) A first example of control

With the first example of control shown in the FIG. 21 flow chart forsetting the predetermined magnification photography mode, onemagnification is set from among a plurality of photographicmagnifications, and zooming drive control is automatically performed soas to keep the magnification as set, even if the object to bephotographed moves.

First, in the step S701, a signal is input to the CPU 61 indicatingwhich photographic magnification is being selected via the controlswitches of the predetermined magnification photographic mode settingunit 72, and thereby the CPU 61 sets that chosen photographicmagnification from among the plurality of available photographicmagnifications. And, based upon the selected photographic magnification,the CPU 61 determines the proportion which the principal object to bephotographed occupies of the photographic field of view. For each of theavailable photographic magnifications, there is recorded in the memoryof the CPU 61 the most suitable proportion for the principal object tobe photographed to occupy in the photographic field of view. Forexample, for the portrait photographic mode, the proportion which theprincipal object to be photographed occupies of the photographic fieldof view is about 30% to 40%. It is appropriate to determine thisproportion by experiment according to the photographic magnificationwhich is to be maintained.

Next, in the step S702, the principal object to be photographed isdesignated within the follow-up area 55b by actuation of the follow-uparea designation member 73. And next, in the step S703, based upon thecharacteristic information for the principal object to be photographeddesignated in the follow-up area 55b, the form of the principal objectto be photographed within the photographic field of view 55a isinferred, and is recognized. The procedure for inference and recognitionof the principal object to be photographed may be suitably selected fromvarious per se known methods; for example, the form of the principalobject to be photographed may be inferred and recognized by grouping itscharacteristic information, or the procedure for its recognition may bethat the characteristic forms of principal objects to be photographed(the angular portions of their contours and the like) may be previouslystored in the memory as templates, so that according to thecharacteristic information a procedure or the like of matching with thetemplates can be implemented. Next, in the step S704, based upon theform of the principal object to be photographed which was inferred andrecognized in the step S703, it is detected to what position in thephotographic field of view 55a the principal object to be photographedhas moved.

Next, in the decision step S705, based upon the form of the principalobject to be photographed which was recognized in the step S703, theproportion which the principal object to be photographed actuallyoccupies of the photographic field of view is calculated, and thiscalculated proportion is compared with the proportion previouslyselected in the step S701 according to the set photographicmagnification. If these proportions are substantially equal, then thissubroutine SR12 terminates and returns control to the main program whoseflow chart is shown in FIG. 19; while, if the proportions are notsubstantially equal, the flow of control proceeds to the step S706.

In this step S706, the zooming lens group 51b of the photographic lens51 is driven in such a manner as to establish the predeterminedphotographic magnification. After this, the flow of control returns tothe step S704.

(b) A second example of control

With the second example of control shown in the FIG. 22 flow chart forsetting the predetermined magnification photography mode, after a zoomposition is designated by the photographer, drive control is performedso as to subsequently maintain the detected photographic magnification.

First, in the step S801, the system waits until the zooming lens group51b of the photographic lens 51 is driven to the position desired by thephotographer, and then the photographic magnification is determined bythis zoom position.

Next, in the step S802, the principal object to be photographed isdesignated within the follow-up area 55b. And next, in the step S803,based upon the characteristic information for the principal object to bephotographed designated in the follow-up area 55b, the form of theprincipal object to be photographed within the photographic field ofview 55a is inferred, and is recognized. Next, in the step S804, basedupon the form of the principal object to be photographed which wasinferred and recognized in the step S803, the proportion which theprincipal object to be photographed actually occupies of thephotographic field of view is calculated.

Next, in the step S805, the position in the photographic field of view55a to which the principal object to be photographed has moved isfollowed up. And next, in the step S806, it is detected whether theproportion which the principal object to be photographed occupies of thephotographic field of view, as calculated in the step S804, has changedaccording to movement of the object to be photographed, or has notchanged. According to the result, if the proportion occupied by theprincipal object to be photographed has not changed, then thissubroutine SR12 terminates and returns control to the main program whoseflow chart is shown in FIG. 19; while, if the proportion occupied by theprincipal object to be photographed has changed, then the flow ofcontrol passes next to the step S807.

In this step S807, the zooming lens group 51b of the photographic lens51 is driven so as to bring the photographic magnification to thepredetermined value. After this, the flow of control returns to the stepS805 again.

(4) Setting the AF mode

Three versions of the details of the subroutine SR13 called from theflow chart of FIG. 19 are shown in the flow charts of FIGS. 23, 24, and25. Since in the flow charts of FIG. 20A and FIG. 20B described abovethe target object follow-up mode was set and operated according to thephotographic mode, therefore it will be explained how the focusdetection mode is controlled, when the target object follow-up mode hasbeen selected, using the flow charts of FIGS. 23 through 25 describedbelow.

(a) A first example of control

The FIG. 23 flow chart shows a first example of control of the AF mode,suitable for a camera which is provided with two AF modes: an S-AF modeand a C-AF mode.

In the first decision step S901, a decision is made as to whether or notthe target object follow-up mode is being selected according to ordersfrom the follow-up area designation member 73, and if in fact the targetobject follow-up mode is being selected then the flow of controlproceeds to the decision step S902, while otherwise this SR13 subroutineterminates. In the decision step S902, a decision is made as to whetheror not the AF mode is being set to the S-AF mode according to ordersfrom the focus detection mode changeover unit 71, and if in fact the AFmode is being set to the S-AF mode then the flow of control proceeds tothe step S903, while otherwise this SR13 subroutine terminates. Next, inthe step S903, the set AF mode is changed over from the S-AF mode to theC-AF mode. In this manner, when the target object follow-up mode isoperating, it is possible always to ensure proper adjustment of thefocus point to the object to be photographed which is being followed upwithin the predetermined focus detection area in the photographic fieldof view. And, finally, this subroutine SR13 terminates and returnscontrol to the main program whose flow chart is shown in FIG. 19.

(b) A second example of control

The FIG. 24 flow chart shows a second example of control of the AF mode,suitable for a camera which is provided with two AF modes, which in thiscase are a C-AF mode and an M-AF mode.

In the first decision step S1001, a decision is made as to whether ornot the target object follow-up mode is being selected according toorders from the follow-up area designation member 73, and if in fact thetarget object follow-up mode is being selected then the flow of controlproceeds to the decision step S1002, while otherwise this SR13subroutine terminates. In the decision step S1002, a decision is made asto whether or not the AF mode is being set to the C-AF mode according toorders from the focus detection mode changeover unit 71, and if in factthe AF mode is being set to the C-AF mode then the flow of controlproceeds to the step S1003, while otherwise this SR13 subroutineterminates. Next, in the step S1003, the set AF mode is changed overfrom the C-AF mode to the M-AF mode. In this manner, when the targetobject follow-up mode is operating, the focus detection area can bealways changed to coincide with the object to be photographed which isbeing followed up, and it is possible to ensure proper adjustment of thefocus point wherever the object to be photographed is located within thephotographic field of view. And, finally, this subroutine SR13terminates and returns control to the main program whose flow chart isshown in FIG. 19.

(c) A third example of control

The FIG. 25 flow chart shows a third example of control of the AF mode,suitable for a camera which is provided with three AF modes: an S-AFmode, a C-AF mode, and an M-AF mode.

In the first decision step S1101, a decision is made as to whether ornot the target object follow-up mode is being selected according toorders from the follow-up area designation member 73, and if in fact thetarget object follow-up mode is being selected then the flow of controlproceeds to the decision step S1102, while otherwise this SR13subroutine terminates. In the decision step S1102, a decision is made asto whether or not the AF mode is being set to the S-AF mode according toorders from the focus detection mode changeover unit 71, and if in factthe AF mode is being set to the S-AF mode then the flow of controlproceeds to the step S1103, while if the AF mode is being set to a modeother than the S-AF mode then the flow of control proceeds to thedecision step S1104. Next, in the step S1103, the set AF mode is changedover from the S-AF mode to the M-AF mode. In this manner, when thetarget object follow-up mode is operating, the focus detection area canbe always changed to coincide with the object to be photographed whichis being followed up, and it is possible to ensure proper adjustment ofthe focus point wherever the object to be photographed is located withinthe photographic field of view. Then this subroutine SR13 terminates.

On the other hand, in the decision step S1104, a decision is made as towhether or not the AF mode is being set to the C-AF mode according toorders from the focus detection mode changeover unit 71, and if in factthe AF mode is being set to the C-AF mode then the flow of controlproceeds to the step S1105, while otherwise this SR13 subroutineterminates. Next, in the step S1105, the set AF mode is changed overfrom the C-AF mode to the M-AF mode. In this manner, when the targetobject follow-up mode is operating, the focus detection area can bealways changed to coincide with the object to be photographed which isbeing followed up, and it is possible to ensure proper adjustment of thefocus point wherever the object to be photographed is located within thephotographic field of view. And, finally, this subroutine SR13terminates and returns control to the main program whose flow chart isshown in FIG. 19.

Now, although the description of the embodiments shown in FIGS. 23through 25 has been made in terms of how the focus detection mode iscontrolled when the target object follow-up mode has been set, as analternative it would also be acceptable for the construction to be suchthat the target object follow-up mode is automatically selected when thefocus detection mode has been set to the C-AF mode or the M-AF mode, ina manner opposite to that practiced in the shown embodiments.

The flow chart shown in FIG. 26 shows another possibility for theoperation of the subroutine SR11 described above, and in this examplethe photographic mode is changed over in accordance with the targetobject follow-up mode.

First, in the first decision step S1201, a decision is made as towhether or not the target object follow-up mode is being selectedaccording to orders from the follow-up area designation member 73, andif in fact the target object follow-up mode is being selected then theflow of control proceeds to the decision step S1202. In the decisionstep S1202, a decision is made as to whether or not the photographicmode is being set to a photographic mode suitable for photography of anobject which is stationary, according to orders from the photographicmode selection unit 70; and if in fact the photographic mode is beingset to a photographic mode suitable for photography of an object whichis stationary, then the flow of control proceeds to the step S1203,while otherwise the flow of control proceeds to the step S1204.

Next, in the step S1203, the set photographic mode is changed over fromthe photographic mode suitable for photography of an object which isstationary to a photographic mode suitable for a moving object. This isdone because the setting of the target object follow-up mode impliesthat the object to be photographed will move within the photographicfield of view, and, since exposure control cannot be performed in amanner suitable for a photography of a moving object when thephotographic mode is one suitable for photography of a stationaryobject, accordingly the photographic mode is automatically changed overto one suitable for photography of a moving object.

The target pursuit device of the present invention has been shown anddescribed in terms of several embodiments thereof, but its details arenot to be considered as being limited by any of the perhaps quitefortuitous details of said shown embodiments or of the drawings. Forinstance, although the present invention has been shown and described asbeing applied to a single lens reflex type camera, this is not to beconsidered as limitative, since the present invention is also suitablefor application, for example, to an imaging device for photography ofmoving images, such as a video camera or the like. Other variations arealso possible. Accordingly, the scope of the present invention is to beconsidered as limited only by the terms of the appended claims, whichfollow.

We claim:
 1. A target follow-up device, having:an imaging device whichoutputs input image data having a plurality of color components; amemory circuit which, based upon the output from said imaging device,records image data of a target to be followed up as reference data; acalculation circuit which calculates the amount of correlation betweensaid input image data and said reference image data; and a positiondetection circuit which, based upon said amount of correlation, detectsthe position of said target; a color selection circuit which variablyselects at least one color from a plurality of colors contained incommon by said input image data and said reference image data prior tothe calculation of said amount of correlation by said calculationcircuit; wherein said calculation circuit calculates said amount ofcorrelation for the color component selected by said color selectioncircuit.
 2. A target follow-up device according to claim 1, whereinsaidimaging device outputs input image data made up from a plurality ofpicture elements; said memory circuit records reference image data madeup from a plurality of picture elements; and said calculation circuitobtains the difference for each individual picture element between saidinput image data and said reference image data, and calculates theminimum residue as being the minimum value of the residue amounts whichresult from summing said differences.
 3. A target follow-up deviceaccording to claim 1, further comprising a strongest color determinationcircuit which sums said reference image data separately for each saidcolor component and detects for which color the result of said summationis the greatest;and wherein said color selection circuit selects thecolor determined by said strongest color determination circuit.
 4. Atarget follow-up device according to claim 1, further comprising:aneighboring region setting circuit which sets a neighboring regionwithin a photographic field which neighbors on the region containingsaid reference image data; and a greatest difference color determinationcircuit which sums said reference image data separately for each saidcolor component, sums said input image data contained within saidneighboring region separately for each said color component, anddetermines for which color the difference between the summation resultfor said reference image data and the summation result for said inputimage data contained within said neighboring region is the greatest; andwherein said color selection circuit selects the color determined bysaid greatest difference color determination circuit.
 5. A targetfollow-up device according to claim 2, further comprising:a neighboringregion setting circuit which sets a neighboring region within aphotographic field which neighbors on the region containing saidreference image data; and: a residue color determination circuit whichcalculates said residue for said reference image data and said inputimage data contained within said neighboring region separately for eachsaid color component, and determines for which color the resultingresidue is an extremum; and wherein said color selection circuit selectsthe color determined by said residue color determination circuit.